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_  Abstract 

Cluster  analysis  is  a  method  for  understanding  the  spatial  arrangement  of 
pointlike  objects.  It  is  practiced  informally  when  stars  are  seen  as  forming 
galaxies  or  grains  in  Aim  are  viewed  as  depicting  an  image.  This  paper  describes 
some  computer  techniques  for  cluster  analysis  of  a  set  of  points  when  distances 
between  the  points  are  known.  In  general  points  that  are  close  together  will  be 
grouped  in  the  same  cluster.  Moreover,  clusters  of  points  can  be  treated  as  sin¬ 
gle  points  and  grouped  into  higher-order  clusters,  thereby  obtaining  a  hierarchi¬ 
cal  arrangement  that  depicts  large-scale  features  and  Ane  detail  as  well.  These 
techniques  have  been  used  in  psychological  studies  of  conceptualization  and 
memory  retention  at  the  University  of  Colorado,  and  the  applications  are  briefly 
reviewed. 


1 .  Introduction 

One  of  the  basic  mechanisms  for  understanding  the  world  is  to  search  for 
groupings  or  arrangements  among  objects  that  are  distinguished  primarily  by 
spatial  location.  Thus  stars  are  seen  as  forming  constellations,  clusters  and 
galaxies.  Grains  in  film  similarly  reveal  an  image  even  when  they  can  be  seen 
individually.  Much  can  be  learned  by  the  recognition  of  groupings  or  clusters 
among  pointlike  objects,  and  techniques  for  automating  this  process  thus  are  of 
interest. 


This  paper  describes  some  computer  techniques  for  cluster  analysis  of  a  set 
of  points  when  distances  between  the  points  are  known  These  techniques  were 
developed  in  connection  with  studies  of  human  conceptualization  and  memory 
retention  at  the  University  of  Colorado.  The  algorithms  and  computer  imple¬ 
mentation  are  described  and  their  usage  in  the  psychological  studies  is  briefly 
reviewed. 


S 


2.  Theory 

In  general  in  a  clustering  problem  we  are  given  a  finite,  nonempty  set  of 
points  5  with  an  implied  distance  relation,  and  are  asked  to  find  a  ''clustering" 
under  which  points  that  are  close  together  will  tend  to  be  in  the  same  grouping 
or  cluster.  Thus  the  mutual  distances  are  regarded  as  defining  a  spatial 
arrangement  of  the  points.  The  purpose  of  clustering  then  is  to  furnish  an 
unambiguous  interpretation  of  the  structure  of  the  spatial  arrangement.  The 
interpretation  in  turn  will  depend  on  the  method  chosen  for  clustering,  but  it 
can  be  hoped  that  significant  features  will  not  depend  strongly  on  the  method 
that  is  used. 

Thus  in  particular  the  notion  of  "cluster"  implies  an  arrangement  or  struc¬ 
turing  of  an  "underlying  set"  of  points  chosen  from  S,  into  a  set  containing  indi¬ 
vidual  points  or  constituent  clusters,  with  the  additional  requirement  that  the 
underlying  sets  of  different  constituent  clusters  must  be  disjoint.  For  formal 
purposes  we  define  clusters  over  S  inductively  as  follows. 

1.  Individual  points  j »  c  S  are  clusters.  The  underlying  set  of  a  point  p  is 
taken  to  be  jp),  and  p  is  then  said  to  be  an  atomic  cluster  and  to  have  order  0. 
All  other  clusters  will  be  nonatomic  and  will  have  order  >0. 
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2.  A  nonatomic  clutter  C  will  have  the  form  of  a  nonempty,  nouingleton 
set  of  clusters  of  lower  order,  which  in  turn  will  be  called  its  conxMtamts .  The 
one  additional  requirement  will  be  that  the  underlying  sets  of  these  coastituents 
mutt  all  be  disjoint.  The  clusters  themselves  will  then  be  said  to  be  diqotnt  The 
order  of  C  is  defined  as  1  +  the  maximum  order  of  its  constituents  iie  under¬ 
lying  set  of  C  is  defined  as  the  union  of  the  underlying  sets  of  its  constituents. 

3  All  of  the  dusters  over  5  are  obtained  by  applying  rules  (1)  and  (2) 

In  summary,  (l)  individual  points  are  clusters,  (2)  any  nonempty,  tonsingle- 
ton  set  of  clusters  whose  underlying  sets  are  disjoint,  is  a  cluster,  and  (3)  these 
are  the  only  clusters.  Thus  a  cluster  is  a  tree  with  the  branches  unotdered  In 
particular  there  are  only  finitely  many  clusters  over  5  If  5  is  a  singleton  then 
the  only  cluster  Is  the  single  member  of  S  (but  not  5  itself).  For  larger  S,  S 
itself  will  be  a  cluster  over  5,  in  addition  to  its  individual  points.  If  5  =  $1,  2,  3J 
the  clusters  over  S  are  1.  2.  3  jl.  2|.  (1.  3(.  |2,  3{.  |1.  2.  3}.  \\i.  2{.  3}.|}1.  3j.  2j. 
\l,  }2.  3||  For  larger  sets  there  are  many  more  clusters 

Given  a  cluster  C  over  S.  a  subcluster  is  defined  inductively  as  either  C,  or 
if  C  is  nonatomic.  one  of  its  constituents  or  a  subcluster  of  one  of  Is  consti¬ 
tuents.  Thus  in  view  of  assumption  (2)  the  underlying  sets  of  two  subdusters  of 
C  must  be  disjoint,  or  one  underlying  set  must  be  included  in  the  other. 

A  clustering  of  5  is  a  cluster  over  S  whose  underlying  set  is  5  Thus  in  the 
above  illustration  there  are  four  clusterings  of  the  three-element  set  5.  and  m 
addition,  six  other  clusters  over  S.  Although  many  clusterings  of  any  sizable  S 
are  possible,  the  meaningful  ones,  from  our  point  of  view,  are  those  in  which 
objects  that  are  close  together  are  grouped  into  clusters  of  low  order.  More¬ 
over.  to  form  higher-order  clusters  in  a  meaningful  way  it  is  necessary  to  extend 
the  measure  of  distance,  assumed  to  be  given  for  the  individual  points,  to  arbi¬ 
trary  clusters.  In  this  way  the  entire  set  5  can  be  formed  into  a  clsster  that 
reveals  large-scale  structure  as  well  as  fine  detail  in  the  spatial  arrangement  of 
points. 

Something  should  be  said  about  the  allowable  measures  of  distance, 
whether  between  points  or  clusters.  In  the  most  restrictive  case  the  distance 
function  d  was  required  to  be  a  metric,  that  is  to  satisfy  the  following  properties 
for  points  (or  clusters)  x.  y  and  z. 

1.  d(z, y)>  0.  with  equality  holding  if  and  only  if  x  =  y. 

8.  d(x,y)  =  d'  ;,x). 

3.  d(x.*)  <  d(x,y)+d  ,*). 

However  these  properties  were  not  always  enforced,  particularly  in  tke  case  of 
nonatomic  clusters.  The  minimum  conditions  that  were  always  enforced  were: 

d(z.y)  it  0,  with  equality  holding  if  x  =  y 

2.  d(x,y)  =  d(y,x). 

That  is,  the  distance  was  always  required  to  be  nonnegative  and  relexive,  or 
independent  of  the  direction  of  measurement. 

The  methods  of  clustering  of  interest  here,  then,  are  distance -based. 
There  are  tiro  main  steps  in  formulating  a  method  of  this  type:  (1)  establishing 
the  distance-based  criterion  under  which  two  objects  (whether  points  or  clus¬ 
ters)  will  always  be  placed  In  the  same  cluster,  and  (2)  extending  the  neasure  of 
distance  as  far  as  necessary,  so  that  a  distance  is  defined  between  any  two  clus¬ 
ters  that  would  be  considered  for  inclusion  in  a  cluster  of  higher  order. 
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Clustenng  then  proceeds  from  the  original,  unstructured  set  of  points 
Points  that  meet  the  necessary  criterion  are  grouped  into  clusters  One  such 
cluster  may  contain  more  than  two  points  because  clustering  is  transitive,  that 
is.  if  a  and  b  are  In  the  same  cluster,  and  b  and  c  are  in  the  same  cluster,  then 
a  and  c  must  be  in  the  same  cluster  In  general  neighboring  points  will  be 
grouped  together  On  the  other  hand  a  cluster  may  consist  of  a  single  point  if  no 
suitable  neighbor  can  be  found  In  any  case,  the  points  are  grouped  into  clus¬ 
ters  as  far  as  the  criterion  allows,  then  the  process  is  iterated,  clusters  being 
grouped  into  clusters  of  higher  order  In  this  manner  the  entire  set  of  points  is 
finally  structured  into  one  large  cluster  1'hus  on  each  iterative  step,  if  there 
are  at  least  two  objects,  that  is,  if  the  final  cluster  has  not  already  been  reached, 
at  least  two  objects  must  be  grouped  into  a  cluster 


3  Methods  for  Distance-Based  Clustering 

In  the  work  reported  here  there  were  two  main  criteria  for  placing  objects 
in  clusters  during  one  iterative  step  of  analysis  In  the  first  version  two  objects 
were  put  in  the  same  cluster  if  either  object  was  a  nearest  neighbor  of  the  other 
In  the  second  version  the  two  objects  had  to  be  mutual  nearest  neighbors  to  be 
guaranteed  placement  in  the  same  cluster  in  either  case  the  distance  measure 
had  to  be  extended  to  higher-order  clusters.  The  main  means  of  doing  this  was 
to  define  the  distance  between  two  clusters  as  the  separation  distance  between 
the  underlying  sets,  that  is.  the  minimum  distance  from  a  point  in  one  underly¬ 
ing  set  to  a  point  in  the  other 

Other  measures  than  the  separation  distance,  which  is  not  a  metric,  were 
used  on  occasion  It  was  found,  however,  that  even  when  the  properties  or  a 
metric  were  enforced  the  results  were  not  much  affected  so  that  the  separation 
distance,  which  is  easy  to  compute,  became  standard 

The  methods  of  clustering,  then,  were  primarily  distinguished  by  how  they 
placed  objects  in  the  same  cluster  during  one  step  of  analysis.  The  first  version 
will  be  referred  to  as  the  nearest  -neighbor  method  and  the  second  (for  reasons 
given  later)  as  contouring  Both  methods  are  illustrated  in  fig.  i,  in  which  a 
hypothetical  array  of  five  points  is  to  be  clustered  based  on  the  usual  Euclidean 
distance  in  the  plane  This  initial  array  is  shown  with  the  points  labeled  in  (la) 
The  successive  steps  for  the  nearest-neighbor  method  are  shown  in  (lb-c)  and 
the  steps  for  contouring  are  shown  in  (Id-f). 

Thus  the  nearest-neighbor  method  takes  only  two  iterative  steps  to  com¬ 
plete  the  clustering,  while  three  are  needed  for  contouring.  Moreover  on  each 
step  of  the  first  method  any  cluster  is  paired  with  at  least  one  other  one  (since  it 
must  have  a  nearest  neighbor),  thus  all  constituents  of  a  cluster  must  have  the 
same  order.  Contouring,  however,  allows  constituents  to  have  differing  order 
Thus  in  the  clustering  of  (1c)  every  constituent  has  order  1  while  in  that  of  (If) 
the  two  constituents  have  orders  0  and  2. 
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Fig  i,  showing  Ihe  successive  stages  of  clustering  for  two  different 
methods  (a)  original  array  of  points;  (b-c)  nearest-neighbor  method;  (d-f)  con¬ 
touring 
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The  two  clusterings,  moreover,  show  significant  differences  in  structure 
Thus  in  C.c)  point  1  is  paired  with  points  2  and  3  in  one  constituent,  while  m  ( ' f  j 
it  is  isolated  as  a  constituent  bv  itself  In  (lo)  on  the  oilier  li.imt,  point-,  i  .in. I 
form  one  constituent  while  in  v  f)  two  clusters  consisting  respectively  of  points 
and  3  and  points  4  and  5,  are  formed  into  one  constituent.  All  this  is  a  conse¬ 
quence  of  the  fact  that  point  1  is  a  considerable  distance  from  the  other  points, 
even  its  nearest  neighbor,  point  3  Thus  the  first  method  is  forced  to  group 
point  I  with  point  3,  regardless  of  the  distance,  while  the  second  method,  by  iso¬ 
lating  point  -  and  forming  the  other  points  into  one  constituent,  is  able  to  give  a 
better  indication  of  the  actual  distances  involved  in  the  spatial  arrangement. 

In  fact  for  the  second  method  each  cluster  of  any  order  has  a  "dispersion 
distance"  such  that  (!)  the  constituents  are  within  this  distance  of  their  nearest 
neighbors  which  are  also  within  the  cluster  and  (2)  any  other,  disjoint  cluster 
must  be  at  a  greater  distance  from  any  constituent  and  thus  from  the  cluster  as 
a  whole  The  given  cluster,  then,  is  contained  in  a  "contour"  drawn  at  the  disper¬ 
sion  distance  around  its  constituents,  and  thereby  is  isolated  from  all  other  dis¬ 
joint  clusters,  thus  the  method  has  been  referred  to  as  "contouring".  In  general 
contouring  yields  a  more  detailed  structure  that  better  reflects  the  large 
disparity  that  may  exist  among  the  nearest-neighbor  distances  Often,  however, 
there  is  a  great  deal  of  fine  structure  for  this  method  so  that  some  "coarsening" 
--  removal  of  contours  -  is  helpful  in  visualizing  the  larger-scale  structure. 


4.  Algorithms 

Both  methods  of  clustering  can  be  carried  out  efficiently,  that  is,  in  time 
that  is  polynomial  in  the  number  of  points  in  the  set,  j  5 1 ,  with  reasonable 
assumptions  about  the  representation  of  the  points  in  »S'  and  the  difficulty  of 
computing  distances  between  them  On  each  iterative  step  of  clustering  we 
must  determine  the  distance  between  clusters;  clearly  this  is  a  polynomial-time 
operation  since  it  depends  only  on  the  distances  between  points  in  the  underly¬ 
ing  sets,  which  are  disjoint  subsets  of  S  (A  polynomial-time  clustering  was  also 
obtained  with  other  definitions  of  the  inter-cluster  distance.)  For  the  nearest- 
neighbor  method  any  cluster  is  joined  with  at  least  one  other  one  on  each  itera¬ 
tive  step,  thus  the  number  of  clusters  is  reduced  by  at  least  half  and  the  total 
number  of  steps  therefore  is  not  more  than  log2jSj.  With  contouring  it  is  possi¬ 
ble  for  only  two  clusters  to  be  joined  on  each  step  so  the  number  of  steps  could 
be  as  large  as  j5j— 1,  but  the  timing  will  still  be  polynomial  in  ;S j.  (In  practice 
it  has  not  been  excessive  compared  with  the  other  method.) 

An  important  feature  of  many  of  the  clustering  problems  to  date  is  that 
many  of  the  point-to-point  distances  are  infinite  (in  practice,  a  very  large 
number)  Thus  a  boolean  relation  is  given  such  that  points  are  a  finite  distance 
apart  or  are  "connected"  only  if  the  relation  holds  between  them  Typically  the 
boolean  c  onnections  are  not  much  more  numerous  than  the  points  themselves 
but  they  always  form  a  connected  graph,  so  that  a  path  of  connections  can  be 
formed  between  a  iy  two  points.  This  means  that  any  reasonable  distance-based 
clustering  will  have  all  constituents  of  any  subcluster  at  finite  distances  from 
their  nearest  neighbors 

The  nearest-neighbor  method  is  implemented  as  follows.  On  each  iterative 
step  an  array  of  clusters  is  formed.  These  clusters  are  to  be  grouped  into  clus¬ 
ters  of  the  next-higher  order.  To  each  cluster  in  the  array  is  associated  its 
"adjacency  list"  --  those  clusters  at  a  finite  distance  from  it  These  clusters  are 
sorted  by  increasing  distance  and  those  at  the  smallest  distance  —  the  nearest 
neighbors  --  are  "marked”  along  with  the  original  cluster,  with  a  number  denot¬ 
ing  the  cluster  of  next-higher  order  into  which  they  will  be  placed.  Meanwhile 
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the  original  cluster  is  placed  on  a  list  assigned  to  this  number  This  list,  in 
effect,  is  the  cluster  of  next-higher  order  that  is  being  formed 

The  marking  itself  proceeds  by  examining  each  cluster  in  the  array  in  turn 
If  a  cluster  has  already  been  marked  with  a  number  ths  number  is  retained, 
otherwise  a  new  number  is  created  Then  the  nearest  nnghbors  of  the  cluster 
art-  examined  Ml  of  these  should  be  marked  with  the  sane  number  as  the  origi¬ 
nal  cluster  Tor  those  that  are.  nothing  is  done  If  any  have  not  yet  been 
marked  at  all,  then  they  are  assigned  the  new  number  and  are  added  to  the 
cluster  list  for  this  number  However,  if  a  cluster  has  alrtady  been  marked  with 
a  different  number,  then  (i)  the  cluster  list  assigned  to  the  old  number  is 
retrieved,  (2)  all  clusters  in  this  list,  together  with  the  neghboring  cluster  itself 
are  marked  with  the  new  number,  13)  all  these  clusters  are  added  to  the  cluster 
list  for  the  new  number,  and  {4)  the  cluster  list  for  the  old  number  is  made 
empty  In  this  way  clusters  of  the  next-higher  order  are  gradually  built  up  and, 
whenever  two  clusters  touch  each  other,  they  are  coalesced  into  one 

W  hen  the  process  is  complete  the  numbers  denoting  the  higher-order  clus¬ 
ters  are  examined  For  those  having  nonempty  lists  tfe  lists  are  retrieved  -- 
these  are  the  clusters  of  next-higher  order.  If  there  is  ortv  one  such  cluster  the 
analysis  is  complete  Otherwise  the  iterative  step  is  repented  with  the  new  clus¬ 
ters 

For  contouring  we  could  use  the  same  approach,  that  is.  form  the  new  clus¬ 
ters  the  same  way  on  each  step,  except  that  now  the  neighboring  clusters  would 
have  to  be  mutual  nearest  neighbors,  in  keeping  with  the  rationale  for  contour¬ 
ing  Instead  we  use  a  different  algorithm  which  sometimes  is  much  faster,  and 
appears  to  be  faster  in  general  This  algorithm  is  recuisive  rather  than  itera¬ 
tive  Initially  it  is  giv  n  list  ot  clusters  to  be  formed  into  t  clustering  If  this  list 
is  a  singleton,  that  is  if  it  has  only  one  element,  then  jjst  the  one  element  is 
returned  If  it  contains  two  elements  or  is  larger  but  al  the  nearest-neighbor 
distances  are  the  same,  then  the  list  itself  is  returned  Otherwise  a  nontrivial 
clustering  is  carried  out 

First  clusters  are  formed  into  an  array  and  to  each  is  associated  an  adja¬ 
cency  list  as  before.  Next  a  "contouring  distance"  is  ch»sen  Currently  this  is 
the  median  of  the  nearest-neighbor  distances  of  all  clusters  in  the  array  (with  a 
given  dts'ance  being  counted  the  number  of  times  it  occurs,  rather  than  just 
once),  but  other  choices  would  also  be  acceptable,  as  is  noted  later  Any  two 
clusters  that  are  not  more  than  this  distance  apart  are  g-ouped  together  into  a 
cluster  of  next-higher  order 

The  grouping  into  clusters  is  done  the  same  way  as  n  the  nearest-neighbor 
method  except  that  different  clusters  are  treated  as  "nearest  neighbors",  in  this 
case,  those  that  are  not  more  than  the  contouring  distance  from  each  other 
Thus  )  not  all  clusters  grouped  together  will  be  nearest  neighbors  and  (2)  not 
all  nearest  neighbors  will  be  grouped  together  A  cluster  that  is  more  than  the 
contouring  distance  from  its  nearest  neighbor  will  remnn  isolated.  When  the 
grouping  into  clusters  is  complete,  then,  there  will  be  seme  newly-formed  clus 
ters  of  higher  order  and  some  isolated  clusters  that  were  not  grouped. 

Next  the  algorithm  is  applied  recursively  to  each  newly-formed  cluster  in 
turn  using  its  constituents  as  the  initial  list  of  clusters  In  this  wav  these  dus¬ 
ters  are  structured  as  they  will  appear  in  the  final  clustering.  Finally  the  algo¬ 
rithm  is  applied  recursively  again,  this  time  to  the  list  containing  all  the  newly 
structured  clusters,  together  with  the  older,  isolated  ones.  This  results  in  the 
final  clustering 

In  particular  it  is  easy  to  show  that,  unless  the  algorithm  is  given  a  single¬ 
ton  list  of  clusters  to  start  with,  this  case  can  nevir  arise.  Instead  the 
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clustering  will  proceed  recursively  until  each  list  to  be  clustered  kas  all  the 
nearest-neighbor  distances  the  same  and  no  further  clustering  is  poisible  For 
the  algorithm  to  work  properly,  however,  the  contouring  distance  must  be 
chosen  so  that  clustering  will  go  to  completion  Any  distance  less  than  the  max¬ 
imum  nearest -neighbor  distance,  and  not  less  than  the  minimum  such  distance 
will  do  though  certain  distances  are  expected  to  resulL  in  faster  clustering 

A  particular  case  that  illustrates  how  timing  depends  on  the  method  is  one 
in  which  the  points,  call  themy,,pz.  ,  p* ,  are  structured  so  that  their  dis¬ 
tances  d(pt.p,,,)  form  a  decreasing  sequence  of  (finite)  values  whle  all  dis¬ 
tances  d[pt ,  p})  for  , i —j  are  infinite.  Thus  ptll  will  be  the  (unique)  nearest 
neighbor  of  pl  whenever  <  i  <  n  but  only  p„  t  and  p„  will  be  mutual  nearest 
neighbors  If  these  are  formed  into  a  (two-constituent)  cluster  then  only  it  and 
Pn  s  of  the  remaining  points  will  be  mutual  nearest  neighbors,  and  so  »n 

Thus  if  we  applied  the  nonrecursive  technique  initially  suggested  for  con¬ 
touring  in  which  only  the  mutual  nearest  neighbors  would  be  grouped  on  each 
iterative  step  only  one  new  cluster  could  be  formed  on  each  step  and  the 
number  of  steps  needed  would  be  n  -  Kach  of  these  m  turn  would  require  time 
proportional  to  the  number  of  clusters  so  the  overall  timing  would  >e  propor¬ 
tional  to  n2 

With  the  recursive  algorithm,  using  the  median  of  the  nearest-naghbor  dis¬ 
tances  as  the  contouring  distance,  the  size  of  the  problem  is  reduced  by  half  for 
each  of  the  two  recursive  calls  that  follow  the  initial  call,  so  the  timng  will  be 
proportional  to  n  logzn  The  timing  will  vary  if  a  different  choice  of  the  contour¬ 
ing  distance  is  used  For  example,  suppose  it  is  chosen  to  be  the  fcth  tmallest  of 
the  nearest-neighbor  distances,  with  k/n  close  to  some  constant  c  between  0 
and  :  (For  the  median  c  would  be  O  b  )  The  timing  can  be  shown  Looe  propor¬ 
tional  to  nlog„7i  where  6  =  \ /  max\c , -c  \ .  so  that  the  ratio  of  the  timing  to 
that,  for  the  median  case  is  log6n/ log2n  This  ratio,  consequently  the  timing,  is 
minimized  for  the  median  case  in  which  b  reaches  its  maximum  of  2 

This  result,  however,  is  problem-dependent  For  other  distance  relations  on 
the  n  points  other  choices  of  contouring  distances  may  give  faster  timngs  Still 
the  recursive  algorithm,  with  some  reasonable  choice  of  the  contouring  dis¬ 
tance,  seems  likely  to  have  better  worst-case  behavior  than  the  iterative  algo¬ 
rithm 

Typically  with  contouring  there  is  much  fine  structure  and  the  order  of  the 
clus'enng  is  much  higher  than  in  the  nearest-neighbor  case.  Thus  it  it  desirable 
to  show  several  versions  of  the  clustering  representing  differing  amounts  of 
"coarsening"  or  removal  of  contours  from  the  original  clustering  Thu  will  make 
the  larger  structures  more  apparent  and  also  will  give  some  indication  of  how 
much  the  clustering  depends  on  small  differences  in  the  point-to-poin'.  distances 
or  how  '  robust '  the  clustering  is  Thus  if  what  is  expected  to  be  a  small  amount 
of  coarsening  in  fact  destroys  most  of  the  structure  then  it  must  have  depended 
on  small  differences  in  the  distances  but  if  most  of  it  survives  then  it  is  more 
robust 

The  rationale  used  for  coarsening  is  to  start  with  the  origma  set  of  n 
points,  their  nearest-neighbor  distances,  and  the  ’’fine-structure”  clustering 
given  by  contouring  Next  a  contouring  distance  d  is  chosen  as  the  fcth  smallest 
of  the  nearest-neighbor  distances  where  k/n  is  close  to  a  fixed  value  c  (Actu¬ 
ally  we  choose  fc  so  that  (fc )/  (n -l)  is  as  close  as  possible  to  c  )  Next  all  the 
nonatomic  subclusters  in  the  clustering  are  examined  "top-down”  --  that  is, 
beginning  with  the  whole  clustering,  proceeding  to  its  constituents,  then  to  their 
constituents  and  so  on  Any  subclusters  in  which  the  dispersion  disUnce  is  not 
more  than  d  are  "flattened”  --  replaced  by  their  underlying  sets  --  and  are  then 
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marked  atomic  From  this  point  on  they  will  be  treated  .is  single  points  and 
will  not  be  further  coalesced  dur  ing  more  advanced  stages  of  the  coarsening 

For  trie  next  stage,  then  the  parti  i!!y-rr.  irsened  clustering  will  be  further 
coarsened  using  the  same  method  However  the  "points"  will  consist  of  the 
"top  level  atomic  '  clusters  -  those  t  hat  .me  atomic  cr  arc  marked  such  and  arc 
not  subclusters  of  any  others  marked  atomic  There  will  be  fewer  such  "points" 
than  on  the  previous  step  Thu  a  nearest  m  ighbor  distances  are  determined  and 
the  contouring  distance  IS  selected  as  before,  using  toe  -lime  Value  of  c  Next 
tin  clustering  is  examined  top-down  again  and  anv  subei asters  that  are  not 
marked  atomic  and  wfiose  d.spersic-ri  distatic  s  not  non-  than  the  contouring 
distance  an  flattened  dowr.  to  tr  ■  top-lc*  i"  atomic  sunrlusters.  and  are 
marked  ■-.ton. is  themselves 

I'he  above  step  i-  iterated  -jult-  t  can  go  no  furtner  that  e'  until  the  con- 
tour  ip*;  distal,  c  equals  the  d.  sperm. n  distance  of  t  hi  entire  clustering  lr.  par- 
In  ular  f  •  tner.  many  cord  our  -  will  m  genera!  b.  removed  leaving  a  coarse, 
st  >■..!  Lure  whin-  me ri  t  '  the  fine  struct  ..re  w.l,  oc  prefer  v<  »i  for  smaller  r  ,  w it  n 
a. I  of  It  bring  retained  ;•  '  rp 

Sample  i  vpenri.cn'.. u  output  fee  m.ih  : r.e  "'  -i'"' ;  -■c.gnnor  method  and 
cent  curing  inch. d. rig  i  mr -cm  :ig.  w:i  bi  h.owi,  i  r ,  'h<  r.evt  section  both 
nn  tnod-  as  implemented  req  tire  evt<.-n*ive  c-t  processing  as  well  as  arrays  of 
‘ i s •  v  it  ;  i.-'i*  w.;  found  to  bt  a  i  enven.ent  p-t ograninimg  language  In  par¬ 

ticular  tnc  I  iSI "  r«  curs. oil  f.sc-lity  wa-  useful  for  ecr. touring  because  it  allowed 
.if. tv  names  mu  tnc  arrays  themselves  to  oc  dynamically  created  and  des- 
tri  ved  • !  :  ur-ivc  e-iiis 

i'pi  or  v ramming  was  dr.nc  on  \  \\  ."Bt")  cornpotf  r  m  '"ran/  i.JSI*  under 

the  \  X  r.perat  i(u;  syst.-n  ,  sec.,;  cr,  times  v<»r;ec  windy  depending  on  the 
pai.rui.e  ippiti  avion  lyptc  lily  Vt-«-  -."-’l  tin  i  wfls  of  the  order  of  '.-2  min  for  a 
set  of  annul  >0  po.nts  with  about  BO  boolean  connect, ons.  so  I  hat  most  of  the 
not  i,t  to  point  distances  wort  infinite  \‘ueh  n.cre  lime  a  pout  20-30  min.  was 
needed  for  this  number  of  points  with  every  distance  finite,  since  this  made  the 
adjacency  '  st-  much  i  mg  or  \o  doubt  the  timing  could  be  improved,  so  far  this 
hit'  not  been  Per  essars 


\oo:c 

A'  p.iecvnt  :  here  arc  four  major  i  w.puter  techniques  for  clustering,  each  in 
:w  -  nr..-  that  is  using  the  nearest  nt-.gnbor  method  and  contouring  making 

eig-  "  ptt  grams  p  all  Ail  were  featru  for  .--Indies  of  human  conceptualization. 
T.rup  and  nu-mory  retention  at  the  t  r.iv.-rsity  of  Colorado  Department  of 
vent  !;>gy  |  in  these  studio.-  model  objects  -.iiu.  is  helicopters  or  cranes  are 

a<-  emoii-u  from  component  parts  Typically  an  object  has  about  50  parts  Each 
part  : s  connected  to  one  or  more  others  Parts  that  are  not  physically  con¬ 
nected  are  considered  to  be  an  infinite  distance  apart,  except  for  a  few  eases  in 
which  >i  finite  distance  implying  a  "r™-- . re t ion"  was  accepted  on  grounds  of  sym¬ 
metry  The  number  of  connections  in  the  object  ranged  from  a  few  percent 
more  to  :>0"  more  than  the  number  of  parts,  thus  most  of  the  parts  had  only  a 
few  parts  connecting 

in  t lie  most  basic  clustering  problem  we  are  given  the  order  of  request  of 
I  tie  parts  when  an  object  is  assembled  and  are  asked  to  find  the  "conceptualiza¬ 
tion"  used  in  making  the  assembly  Essentially  the  conceptualization  is  a 
hierarchical  subdivision  of  the  object  in  which  the  individual  connected  parts  are 
grouped  into  larger  connected  units,  and  these  into  still  larger  units,  and  so  on 
In  short  it  is  a  distance-based  clustering  of  the  parts  Parts  that  are  connected 
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and  whoso  orders  of  request  are  nearly  equal  are  assumed  to  belong  near  each 
other  in  the  clustering,  that  is.  as  part  of  the  same  subeluster  of  low  order  'I’he 
distance  between  connected  parts  then,  is  just  the  absolute  value  of  the 
difference  in  tne  orders  of  request  Thus  the  parts  are  treated  as  individual 
points  and  l he  clustering  is  formed  One  variation  of  this  analysis  is  to  obtain  a 
"consensus"  conceptualization  by  averaging  the  orders  of  request  over  several 
assembly  trials 

A  second  problem  concerns  the  clusterings  that  are  obtained  by  the  above 
analysis  when  a  number  of  subjects  all  assemble  the  same  objecL  under  varying 
conditions  We  would  like  to  know  if  these  clusterings  fall  into  meaningful  pat¬ 
terns  or  hierarchies  For  instance  we  can  ask  whether  there  is  essentially  one 
conceptualization  with  minor  variations  or  several  essentially  different  concep¬ 
tualizations  Thus  a  clustering  of  the  conceptualizations  is  called  for 

To  do  this  we  must  define  a  distance  between  eonceptualizations.  The  way 
this  has  been  done  is  to  consider  the  boolean  relation  or  set  of  connections  in 
the  object  The  connections  represented  by  pairs  of  parts,  are  numbered 
",  2.  m  Kach  connection  is  represented  by  a  pair  Jp,,  p2j  Given  any  clus¬ 
tering  there  will  exist  a  minimal  subeluster  ('  that  contains  both  p ,  and  p2,  so 
that  p  i  and  p-.  will  be  in  different  constituents  of  C 

Next  we  define  the  order  of  )p s.  p2j  as  the  order  of  C  This  is  the  “bottom- 
up"  order  since  it  depends-  or,  the  order  of  the  c  onstituents  of  C  An  alternative 
is  tfie  top-down"  order  which  is  tne  depth  of  ('  within  the  clustering,  where  the 
clustering  itself  will  have  depth  zero  its  constituents  depth  one.  their  consti¬ 
tuents  depth  two  and  so  on  M  any  rate  for  any  clustering  we  obtain  an  m- 
vector  «/z.  em)  where  v.  is  the  order  of  the  /th  connection  It  is  not 

difficult  to  show  that  an  entire  clustering  can  be  reconstructed  from  its  associ¬ 
ated  vector  ( For  example  we  can  -tart  with  the  pairs  of  lowest  bottom-up  or 
highest  lop-down  order  arid  add  pairs  of  succeeding  orders  until  the  entire 
arrangement  of  pairs  is  determined  )  Thus  two  clusterings  with  the  same  vector 
must  be  identical 

Finally,  the  distance  between  two  clusterings  is  defined  as  the  distance 
between  the  associated  vectors,  using  a  metric  on  an  m -dimensional  vector 
space  The  metric  that  has  been  most  useful  is  the  Fuchdean  1-norm,  that  is, 
the  sum  of  the  absolute  values  of  the  differences  of  the  corresponding  terms  of 
two  vectors  The  bottom-up  order  and  the  '.-norm  distance  were  used  exten¬ 
sively  in  studies  based  on  the  nearest-neighbor  method  of  clustering  ■  ].  For 
contouring,  however,  the  top-down  order  is  probably  more  appropriate  since  the 
clusterings  arc  not  "fiat-bottomed  ",  that  is,  constituents  in  a  subcluster  can 
have  variable  order  This  means  that  the  bottom-up  order  of  a  connecting  pair 
can  depend  on  a  constituent  that  contains  neither  point  of  the  pair,  something 
that  does  not  occur  with  the  top-down  order  1  he  latter,  then,  seems  to  offer  a 
more  reasonable  indication  of  how  "similar"  the  components  of  a  connection  arc 

It  should  be  noted  that  in  this  sort  of  analysis,  regardless  of  how  the  order 
is  measured,  all  distances  between  points  (in  this  case,  clusterings)  are  finite 
Thus  there  arc  large  adjacency  lists  for  the  points  and  extensive  computation  is 
required  for  a  moderately-sized  problem,  about  2b  min  of  CPU  time  on  the  VAX 
computer  was  needed  for  one  typical  case  involving  47  points 

Thus  far  two  techniques  for  clustering  problems  have  been  described,  both 
dealing  with  cases  in  which  the  objects  were  correctly  assembled  Two  other 
techniques  have  been  developed  for  understanding  what  is  involved  when  the 
assembly  is  incorrect  (Incorrect  assembly  occurred  when  objects  were  recon¬ 
structed  from  memory,  without  a  correctly  assembled  object  as  reference  ) 
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In  the  first  case  we  are  trying  to  determine  how  the  different  connections  in 
the  object  influence  the  errors  that  are  made  Thus  the  connections  are  treated 
as  the  "points  ‘  in  a  cluster  analysis  Two  such  connections  are  connected  them¬ 
selves  if  they  have  an  endpoint  in  common  Thus  a  boolean  relation  can  be 
defined  on  the  set  of  connections  I'wo  connections  that  are  not  connected  to 
each  other  are  at  an  infinite  distanc  e  apart;  otherwise  the  distance  will  depend 
on  how  errors  are  made  at  these  connections  when  assembl>  . '  the  object  is 
attempted 

Thus  it  is  assumed  that,  whenever  the  object  is  incorrectly  assembled, 
there  is  a  way  of  unambiguously  deciding  at  which  connections  the  errors  are 
made  For  the  analysis,  then,  a  number  of  cases  of  an  incorrectly-assembled 
object  ari'  considered  To  define  l  he  distance  between  two  connections  in  the 
object  that  have  an  endpoint  in  common  we  consider  the  sets  .V|  and  Sz  of  cases 
in  which  errors  were  made  on  the  first  or  second  connection,  respectively  The 
distance  is  then  defined  as  S,-Sz./  St  ^jS2 j .  that  is,  as  the  ratio  between  the 
number  of  points  in  the  symmetric  difference  of  the  two  sets  (where  the  sym¬ 
metric  difference  is  the  set  of  points  in  the  union  that  are  not  in  the  intersec¬ 
tion)  and  the  number  of  points  in  the  union  This  distance  can  be  shown  to  be  a 
metric  on  finite,  nonempty  sets  2]  In  particular  the  distance  is  always  <*. .  it  is 
set  to  '  by  default  if  >>,  is  empty 

The  distance  is  smaller  if  there  is  greater  agreement  between  the  errors 
made  on  one  connection  and  those  on  the  other  'llnus  the  clustering  will  tend  to 
identify  groups  of  connections  for  which  the  pattern  of  errors  is  similar,  such  as 
those  on  which  nearly  all  subjects  made  errors  In  this  way  it  is  hoped  that  con¬ 
nections  that  are  consistently  troublesome  can  be  identified  so  that  instruc¬ 
tional  sequences  can  be  designed  to  reduce  the  incidence  of  errors 

The  second  technique,  which  is  the  final  one  considered  here,  is  concerned 
with  identifying  meaningful  patterns  of  error  making  among  different  subject 
groups  Subjects  within  a  group  assemble  the  object  under  similar  conditions. 
For  example,  visual  instructional  material  may  be  presented  For  each  subject 
within  a  group,  then,  we  have  the  list  of  connections  on  which  errors  were  made 
For  any  group  we  define  an  m -vector  ( v  ,.  v2.  ,  vm)  by  vt  =  average  number  of 

errors  made  on  the  ith  connection  =  number  of  subjects  who  made  the  errors 
divided  by  the  number  of  subjects  in  the  group  The  distance  between  groups  is 
then  defined  a?  the  distance  between  vectors,  measured  as  before,  and  cluster¬ 
ing  can  proceed  .is  in  the  earlier  case 

An  illustration  of  this  technique  is  shown  in  fig  2  The  twelve  points  in  each 
clustering  represent  twelve  subject  groups  The  object  in  this  case  has  bb  con¬ 
nect,  ons  Thus  each  point  represents  a  loration  in  a  bb-dimensionai  vector 
space  and  the  point-to-point  distances,  which  were  given  by  the  i-norm,  are  only 
roughly  indicated  in  the  figure.  (2a)  shows  the  clustering  obtained  by  the 
no. crest -neighbor  method,  while  (2b-d)  show  the  results  of  contouring  with 
increasing  amounts  of  coarsening  Thus  (2b)  has  no  coarsening,  (2c)  has  a 
moderati  amount,  with  the  coarsening  parameter  c-0f),  and  (2d)  has  the  larg¬ 
est  amount,  with  Certainly  the  two  methods  show  differences  but  there  are 

basic  similarities  too  In  particular  the  nearest-neighbor  method  seems  to 
correspond  best  to  contouring  with  a  moderate  amount  of  coarsening. 
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1  Steinheiser,  Frederick 

0  345 

1  Sympson,  Brad 

1  5 

1  Van  Mat re,  Nick 

3 

1  Wegman,  Dr.  Edward 

1 

1  WEISSINGER-BAYLON  ROGER 

2345 

1  Wetzel,  Douglas  (NPRDC) 

012  45 

1  Wolfe,  John  H. 

0123  5 
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Army  Air  Force 


1  ARI  Technical  Director 

12345 

1  AFHRL  Technical  Documents 

C  nl2345 

1  FARR  BEATRICE 

2345 

1  AFOSR  Life  Sciences 

12345 

1  KATZ  MILTON  S 

12345 

1  Air  University  Library 

12345 

1  O'Neil,  Harry 

345 

1  Christal ,  Raymond  E. 

012345 

1  Orasanu,  Judith 

345 

1  HADDAD  GENEVIEVE 

2345 

1  Psotka,  Joseph 

345 

1  Longridge,  Tom 

0  2345 

1  SASMOR  ROBERT 

12345 

1  PENNELL,  ROGER 

1  5 

1  WISHER  ROBERT 

12345 

1  REE  MALCOLM 

12  45 

1  Yasatuke,  Joe 

2345 

Department  of  Defense 

Civilian  Agencies 

12 

DTIC 

12345 

1  CHIPMAN  SUSAN 

2345 

1 

OUSDRE 

12345 

1  Selden,  Ramsay 

1 

Thorpe,  Jack  A. 

345 

1  USCG  PSYCHOLOGICAL  RESEARCH 

12345 

1  Weiss,  Edward  C. 

1  Young,  Joseph  L.  012345 
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Andersen,  Erling  B. 

12345 

1  Reckase,  Mark  D. 

01  3  5 

Annett,  John 

0  2345 

1  RESNICK  LAUREN 

2345 

Australian  DoD  Psychology 

12345 

1  ROSE  /iNDREW  M 

12345 

Baddeley,  Alan 

2345 

1  Rothkopf,  Ernst 

012345 

Be jar,  Isaac 

12  5 

1  Rouse,  Wi Ilian 

345 

Birenbaun,  Menucha 

1  345 

1  RUDNER  LAWRENCE 

1  5 

Bund,  dcr  Verteidigung 

012  45 

1  RUMELHART  DAVID 

5 

CARPENTER  PAT 

2  45 

1  SAM EJ IMA  FUMIKO 

12345 

Carroll,  John  B. 

0  2  5 

1  Samet ,  Michael 

345 

CHASE  WILLIAM 

2  45 

1  Schneider,  Walter 

0  2345 

CHI  MICHEL1NE 

45 

1  SEIDEL  ROBERT  J 

123  5 

COOPER  LYNN  A 

2  45 

1  Sinaiko,  H.  Wallace 

0  345 

CROSS  KENNETH 

12345 

1  SMITH  EDWARD 

2  45 

ERIC 

12345 

1  SNOW  RICHARD 

12345 

Feltovich,  Paul 

345 

1  Spoehr,  Kathryn  T. 

2345 

Feurzeig,  Wallace 

0  345 

1  Sternberg,  Robert 

0  2345 

FIELDS  VICTOR 

1  3  5 

1  Stevens,  Albert 

0  2345 

Fletcher,  Dexter 

12345 

1  Stone,  David 

01  345 

FREDERIKSEN  JOHN  R 

12345 

1  Stout,  William 

1  5 

Centner,  Don 

2  4 

1  SUPPES  PATRICK 

12345 

Claser,  Robert 

012345 

1  SWAM I NATHAN  HARIHARAN 

12345 

Clock,  Marvin  D. 

0  2345 

1  Tatsuoka,  Kikumi 

012345 

GOCUEN ,  JOSEPH 

2  45 

1  Tatsuoka,  Maurice 

1  34 

Gopher,  Daniel 

2  45 

1  Thissen,  David 

01  5 

Green,  Bert 

012345 

1  Thorndyke,  Perry 

0  345 

CREENO  JAMES  C 

12345 

1  Towne ,  Doug 

0  345 

l^ayes-Koth ,  Barbara 

2345 

1  TSUTAKAWA  ROBERT 

1  5 

Hayes-Roth,  Frederick 

345 

1  Vale,  David 

01  5 

HUMPHREYS  LLOYD 

12  5 

1  Weiss,  David  J. 

012  5 

Hunt,  Earl 

0  2  45 

1  Wescourt ,  Keith  T. 

0  345 

Just ,  Marcel 

2  4 

1  WHITELY  SUSAN 

12  5 

KEELE  STEVEN  W 

2  45 

K I ERAS  DAVID 

2345 

KINTSCH  WALTER 

45 

Kosslyn,  Stephen 

012  45 

Langley,  Patrick 

345 

Lansman,  Marcy 

0  2  5 

Lesgold,  Alan 

012345 

LEVINE  MICHAEL 

12345 

LEWIS  CHARLES 

12345 

LINN,  MARCIA  C. 

0 

LINN  ROBERT 

12  5 

Lord,  Frederic  M. 

01  5 

McClelland,  Jay 

2  45 

Miller,  Janes  R. 

345 

MUNRO  ALLEN 

2345 

Norman,  Donald  A. 

0  2345 

NOVICK  MELVIN  R 

12345 

Orlansky,  Jesse 

012345 

PAULSON  JAMES  A 

12345 

PELLEGRINO  JAMES 

2  45 

PENNINGTON,  NANCY 

345 

Petrullo,  Luigi 

012345 

POLSON  MARTHA 

2345 

POLSON  PETER 

2345 
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Navy 

1  Dr.  Meryl  S.  Baker 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Dr.  Alvah  Bittner 

Naval  Biodynaaics  Laboratory 
New  Orleans,  LA  70189 

1  Liaison  Scientist 

Office  of  Naval  Research 
Branch  Office,  London 
Box  39 

FPO  New  York,  NY  09510 

1  Lt .  Alexander  Bory 
Applied  Psychology 
Measurement  Division 
NAMR1. 

NAS  Pensacola.  FL  32508 

1  Dr.  Richard  Cantone 

Navy  Research  Laboratory 
Code  7510 

Washington,  DC  20375 

I  Chief  of  Naval  Education  and  Training 
Liason  Office 

Air  Force  Human  Resource  Laboratory 
Operations  Training  Division 
WILLIAMS  AFB ,  AZ  85224 

1  Dr.  Stanley  Collyer 

Office  of  Naval  Technology 
800  N.  Quincy  Street 
Arlington,  VA  22217 

1  CDR  Mike  Curran 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Code  270 

Arlington,  VA  22217 

1  Dr.  Tom  Duffy 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  DR.  PAT  FEDERICO 
Code  PI  3 
NPRDC 

San  Diego,  CA  92152 

1  Dr.  John  Ford 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 


Navy 

1  Dr.  Mike  Gaynor 

Navy  Research  Laboratory 
Code  7510 

Washington,  DC  20375 

1  Dr.  Jim  Hoi lan 
Code  304 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Dr.  Ed  Hutchins 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Dr.  Norman  J.  Kerr 

Chief  of  Naval  Technical  Training 
Naval  Air  Station  Memphis  (75) 
Millington,  TN  38054 

1  Dr.  Peter  Kincaid 

Training  Analysis  &  Evaluation  Group 
Dept,  of  the  Navy 
Orlando.  FL  32813 

1  Dr.  William  L.  Maloy  (02) 

Chief  of  Naval  Education  and  Training 
Naval  Air  Station 

Pensacola,  FL  32508 

1  Dr  William  Montague 
NPRDC  Code  13 
San  Diego,  CA  92152 

1  Library,  Code  P201L 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

1  Technical  Director 

Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 

6  Commanding  Officer 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

1  Office  of  Naval  Research 
Code  433 

800  N.  Ouincy  SStreet 
Arlington,  VA  22217 
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Navy 

6  Personnel  &  Training  Research  Group 
Code  442PT 

Office  of  Naval  Research 
Arlington.  VA  22217 

1  Office  of  the  Chief  of  Naval  Operations 
Research  Development  &  Studies  Branch 
OP  115 

Washington,  DC  20350 

1  LT  Frank  C.  Petho,  MSC,  USN  (Ph.D) 

CNET  (N-432) 

NAS 

Pensacola,  FL  32508 

1  Dr.  Gary  Poock 

Operations  Research  Department 
Code  55PK 

Naval  Postgraduate  School 
Monterey,  CA  93940 

J  Dr.  Cil  Ricard 
Code  N711 
NTEC 

Orlando,  FL  32813 

1  Dr.  Bernard  Rimland  (01C) 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

1  Dr.  Carl  Ross 
CNET-PDCD 
Building  90 

Great  Lakes  NTC,  1L  60088 

l  Dr.  Worth  Scanland 
CNET  (N-5) 

NAS,  Pensacola,  FL  32508 

1  Mr.  Irving  Schiff 
Dept .  of  the  Navy 
Chief  of  Naval  Operations 
OP  113 

Washington,  DC  20350 

1  Dr.  Sam  Schiflett 

Aircrew  Systems  (SY70E) 

Naval  Air  Test  Center 
Patuxent  River,  MD  20670 

I  Dr.  Robert  C.  Smith 

Office  of  Chief  of  Naval  Operations 
OP-987H 

Washington,  DC  20350 


Navy 

1  Dr.  Alfred  F.  Smode,  Director 
Training  Analysis  &  Evaluation  Group 
Dept,  of  the  Navy 
Orlando,  FL  32813 

1  Dr.  Richard  Sorensen 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

1  Dr.  Frederick  Steinheiser 
CNO  -  OP l 1 5 
Navy  Annex 
Arlington,  VA  20370 

1  Mr.  Brad  Sympson 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

1  Code  14 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

1  Dr.  Edward  Wegman 

Office  of  Naval  Research  (Code  411SAP) 
800  North  Quincy  Street 
Arlington,  VA  22217 

1  Roger  Weissinger-Baylon 

Department  of  Administrative  Sciences 
Naval  Postgraduate  School 
Monterey,  CA  93940 

1  Dr.  Douglas  Wetzel 
Code  12 

Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 

1  Mr  John  H.  Wolfe 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152 
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Marine  Corps 

1  H.  William  Greenup 

Education  Advisor  (E031) 
Education  Center,  MCDEC 
Quant i co,  VA  22134 

1  Special  Assistant  for  Marine 
Corps  Matters 
Code  100M 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Arlington,  VA  22217 

1  DR.  A.L.  SLAFKOSKY 

SCIENTIFIC  ADVISOR  (CODE  RD-1) 
HQ,  U.S.  MARINE  CORPS 
WASHINGTON,  DC  20380 


Army 

1  Technical  Director 

U.  S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Beatrice  J.  Farr 

U.  S.  Army  Research  Institute 
5001  Elsenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Milton  S.  Katz 
Training  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Harold  F.  O'Neil,  Jr. 

Director,  Training  Research  Lab 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Commander,  U.S.  Army  Research  Institute 
for  the  Behavioral  &  Social  Sciences 
ATTN:  PERI-BR  (Dr.  Judith  Orasanu) 

5001  Eisenhower  Avenue 
Alexandria,  VA  20333 

1  Joseph  Psotka,  Ph.D. 

ATTN:  PERI-1C 

Army  Research  Institute 

5001  Eisenhower  Ave. 

Alexandria,  VA  22333 

1  Dr.  Robert  Sasmor 

U.  S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

I  Dr.  Robert  Wisher 

Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 
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Air  Force 

1  Technical  Documents  Center 
Air  Force  Human  Resources  Laboratory 
WPAFB,  OH  45433 

1  U.S.  Air  Force  Office  of  Scientific 
Research 

Life  Sciences  Directorate,  NL 
Bolling  Air  Force  Base 
Washington,  DC  20332 

1  Air  University  Library 
AUL/LSE  76/443 
Maxwell  AFB ,  AL  36112 

1  Mr.  Raymond  E.  Christal 
AFHRL/MOE 

Brooks  AFB.  TX  78235 

l  Dr.  Genevieve  Haddad 
Program  Manager 
Life  Sciences  Directorate 
AFOSR 

Bolling  AFB.  DC  20332 

1  Dr.  T.  M.  Long ridge 
AFHRL/OTE 

Williams  AFB.  AZ  85224 

1  Dr.  Roger  Pennell 

Air  Force  Human  Resources  Laboratory 
Lowry  AFB,  CO  80230 

1  Dr.  Malcolm  Ree 
AFHRL/MP 

Brooks  AFB,  TX  78235 


Department  of  Defense 

12  Defense  Technical  Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Military  Assistant  for  Training  and 
Personnel  Technology 

Office  of  the  Under  Secretary  of  Defens 
for  Research  4  Engineering 
Room  3D129,  The  Pentagon 
Washington,  DC  20301 

1  Major  Jack  Thorpe 
DARPA 

1400  Wilson  Blvd. 

Arlington,  VA  22209 


1  Dr.  Joseph  Yasatuke 
AFHRL/LRT 

Lowry  AFB,  CO  80230 
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1  Dr.  Susan  Chipaan 
Learning  and  Development 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1  Dr.  Ramsay  W.  Selden 

National  Institute  of  Education 
1200  19th  St.,  NW 
Washington,  DC  20208 

1  Chief,  Psychological  Reserch  Branch 
U.  S.  Coast  Guard  (G-P-1/2/TP42) 
Washington,  DC  20593 

I  Dr.  Edward  C.  Weiss 
National  Science  Foundation 
1800  C  Street,  NW 
Washington,  DC  20550 


1  Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Stud lest raede  6 
1455  Copenhagen 
DENMARK 

1  Dr.  John  Annett 

Department  of  Psychology 
University  of  Warwick 
Coventry  CV4  7AJ 
ENGLAND 

1  Psychological  Research  Unit 
Dept,  of  Defense  (Army  Office) 
Campbell  Park  Offices 
Canberra  ACT  2600 
AUSTRALIA 

1  Dr.  Alan  Baddeley 

Medical  Research  Council 
Applied  Psychology  Unit 
15  Chaucer  Road 
Cambridge  CB2  2EF 
ENGLAND 

1  Dr.  Isaac  Bejar 

Educational  Testing  Service 
Princeton,  NJ  08450 

1  Dr.  Menucha  Birenbaum 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Ramat  Aviv  69978 
Israel 

1  Bundminlsterium  der  Verteidigung 
-Referat  P  II  4- 
Psychological  Service 
Postfach  1328 
D-5300  Bonn  1 
F.  R.  of  Germany 

1  Dr.  Pat  Carpenter 

Department  of  Psychology 
Carnegi e-Mel  Ion  University 
Pittsburgh,  pa  152H 

1  Dr.  John  P.  Carroll 
4PU  El  1 iott  Rd. 

Chapel  Hill,  t.f  7/514 
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Private  Sector 

1  Or.  William  Chase 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Micheline  Chi 
Learning  R  &  D  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  Lynn  A.  Cooper 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  Kenneth  B.  Cross 
Anne a pa  Sciences,  Inc. 

P.0.  Drawer  0 

Santa  Barbara,  CA  93102 

1  ERIC  Facility-Acquisitions 
483^  Rugby  Avenue 
Bethesda,  MD  20014 

1  Dr.  Paul  Feltovich 

Department  of  Medical  Education 
Southern  Illinois  University 
School  of  Medicine 
P.0.  Box  3926 
Springfield,  IL  62708 

1  Mr.  Wallace  Feurzeig 

Department  of  Educational  Technology 
Bolt  Beranek  &  Newman 
10  Moulton  St. 

Cambridge,  MA  02238 

1  Dr.  Victor  Fields 
Dept .  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

1  Dr.  Dexter  Fletcher 
WICAT  Research  Institute 
1875  S.  State  St. 

Orem.  UT  22333 

I  Dr.  John  R.  Frederiksen 
Bolt  Beranek  &  Newman 
50  Moulton  Street 
Cambridge,  MA  02138 


Private  Sector 
1  Dr.  Don  Centner 

Center  for  Human  Information  Processing 
University  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Robert  Claser 

Learning  Research  &  Development  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
PITTSBURGH,  PA  15260 

1  Dr.  Marvin  D.  Clock 
217  Stone  Hall 
Cornell  University 
Ithaca,  NY  14853 

1  Dr.  Josph  Goguen 
SRI  International 
333  Ravenswood  Avenue 
Menlo  Park,  CA  94025 

1  Dr.  Daniel  Copher 

Department  of  Psychology 
University  of  Illinois 
Champaign,  IL  61820 

1  Dr.  Bert  Green 

Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

1  DR.  JAMES  G.  GREENO 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O'HARA  STREET 
PITTSBURGH,  PA  15213 

1  Dr.  Barbara  Hayos-Roth 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  95305 

1  Dr.  Frederick  Hayes-Roth 
Teknowledge 
525  University  Ave. 

Palo  Alto,  CA  94301 

1  Dr.  Lloyd  Humphreys 
Department  of  Psychology 
University  of  Illinois 
Champaign,  IL  61820 
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l  Ur.  Earl  Hunt 

Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  98105 

1  Dr.  Marcel  Just 

Department  of  Psychology 
Carnegi e-Mel Ion  University 
Pittsburgh,  PA  15213 

1  Dr.  Steven  W.  Keele 
Dept,  of  Psychology 
University  of  Oregon 
Eugene.  OR  97403 

1  Dr.  David  Kieras 

Department  of  Psychology 
University  of  Arizona 
Tuscon,  AZ  85721 

1  Dr.  Walter  Ktntsch 

Department  of  Psychology 
University  of  Colorado 
Boulder,  CO  80302 

1  Dr.  Stephen  Kosslyn 

Department  of  Psychology 
The  Johns  Hopkins  University 
Baltimore,  MD  21218 

1  Dr.  Pat  Langley 

The  Robotics  Institute 
Carnegie-Mellon  University 
Pittsburgh.  PA  15213 

1  Dr.  Marcy  Lansman 

The  L.  L.  Thurstone  Psychometric 
Laboratory 

University  of  North  Carolina 
Davie  Hall  013A 
Chapel  Hill,  NC  27514 

1  Dr.  Alan  Lesgoid 
Learning  R6D  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Michael  Levine 

Department  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 


1  Dr.  Charles  Lewis 

Faculteit  Sociale  Wetenschappen 
Rijksuniversiteit  Groningen 
Oude  Boteringestraat  23 
9712GC  Croningen 
Netherlands 

1  Dr.  Marcia  C.  Linn 

University  of  California 

Director,  Adolescent  Reasoning  Project 

Berkeley,  CA  94720 

1  Dr.  Robert  Linn 

College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

1  Dr.  Frederic  M.  Lord 

Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  Jay  McClelland 

Department  of  Psychology 
MIT 

Cambridge,  MA  02139 

1  Dr.  James  R.  Miller 

Computer*Thought  Corporation 
1721  West  Plano  Highway 
Plano,  TX  75075 

1  Dr.  Allen  Munro 

Behavioral  Technology  Laboratories 
1845  Elena  Ave.,  Fourth  Floor 
Redondo  Beach,  CA  90277 

1  Dr.  Donald  A  Norman 

Cognitive  Science,  C-015 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Melvin  R.  Novick 

356  Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  Jesse  Orlansky 

Institute  for  Defense  Analyses 
1801  N.  Beauregard  St. 

Alexandria,  VA  22311 
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Private  Sector 

1  Dr.  James  A.  Paulson 

Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 

1  Dr.  James  W.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  Barabara,  CA  93106 

1  Dr.  Nancy  Pennington 
University  of  Chicago 
Craduate  School  of  Business 
1101  E.  58th  St. 

Chicago,  IL  60637 

1  Mr.  L.  Petrullo 
3695  N.  Nelson  St. 

ARLINGTON,  VA  22207 

l  Dr.  Martha  Poison 

Department  of  Psychology 
Campus  Box  346 
University  of  Colorado 
Boulder,  CO  80309 

1  DR.  PETER  POLSON 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  COLORADO 
BOULDER,  CO  80309 

1  Dr.  Mark  D.  Recknse 
ACT 

P.  0.  Box  168 
Iowa  City,  IA  52243 

1  Dr.  Lauren  Resnick 
LRDC 

University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  1521 

1  Dr.  Andrew  M.  Rose 

American  Institutes  for  Research 
1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 

1  Dr.  Ernst  7..  Rothkopf 
Bell  Laboratories 
Murray  Hill,  NJ  07974 


Private  Sector 

1  Dr.  William  B.  Rouse 
Ceorgia  Institute  of  Technology 
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